[DON'T MERGE] Run multi-rank CPU unit tests in CI via LOCAL_SIZE - #8381

…dai#8398)

## Problem

`TestMultipleModels::test_zero_optimizer`, `TestSimpleMoE`, `TestMoE`,
`TestPRMoE`, and `TestMOETensorParallel` hardcode `"fp16": {"enabled":
True}` in their DeepSpeed configs. The engine's sanity check then
raises:

```
ValueError: Type fp16 is not supported on your device.
```

on any accelerator whose `is_fp16_supported()` is false. On CPU that
maps to the AVX512-FP16 capability of the host, and GitHub's
`ubuntu-24.04` runners are hardware-heterogeneous: **the same test
passes on one runner and fails on the next** (observed directly in deepspeedai#8381
— 146 failures appeared on one runner generation and none on another,
with identical code).

## Change

Skip these tests via a capability query:

```python
@pytest.mark.skipif(not get_accelerator().is_fp16_supported(), reason="fp16 is not supported on this accelerator")
```

- capability only, no accelerator-name matching;
- mirrors the existing bf16 skip precedent in
`tests/unit/v1/zero/test_zero_user_backward.py`;
- deliberately a **skip** rather than silently running bf16 — these
tests exist to cover the fp16 paths.

## Validation

Validated as part of the multi-rank CPU CI experiment in deepspeedai#8381: the 146
hardware-lottery failures became deterministic skips, zero regressions
on previously-passing tests.

Signed-off-by: Guokai Ma <guokai.ma@intel.com>

## Problem

`tests/unit/v1/zero/test_zero_user_backward.py` builds torch DDP
**reference** models (the known-good baseline that ZeRO results are
compared against) at five sites:

```python
model_ddp = DDP(model_ddp, device_ids=[rank], output_device=rank)
```

`device_ids=[rank]` assumes rank ↔ GPU index. torch's DDP contract only
allows `device_ids`/`output_device` for single-device GPU modules; **CPU
modules live on one shared device and must omit them**, so multi-rank
CPU runs died inside the DDP constructor with:

```
ValueError: DistributedDataParallel device_ids and output_device arguments only work with single-device/multiple-device GPU modules or CPU modules, ...
```

## Change

Route all five sites through one helper:

```python
def wrap_ddp_reference(model, device, rank):
    # Only indexed devices take device_ids/output_device; CPU modules live on one shared device.
    if torch.device(device).type == 'cpu':
        return DDP(model)
    return DDP(model, device_ids=[rank], output_device=rank)
```

Design notes:

- the condition restates the exact precondition torch's own DDP
constructor enforces, in torch's device vocabulary — it follows the
model's actual device rather than the global accelerator configuration;
- a single helper means new reference-model sites cannot forget the
branch (the first fix round in deepspeedai#8381 missed 4 of the 5 sites for exactly
this reason);
- GPU behavior is unchanged.

## Validation

Validated as part of the multi-rank CPU CI experiment in deepspeedai#8381: all
DDP-constructor failures were eliminated (the file's few remaining
failures there are unrelated — see the triage table in that PR), zero
regressions vs the same-commit baseline.

Signed-off-by: Guokai Ma <guokai.ma@intel.com>

…eepspeedai#8397)

## Problem

`get_accelerator().current_device()` returns a **device index** on GPU
backends (`torch.cuda.current_device()` → int), but on CPU it returns
the `LOCAL_RANK` environment value — a plain **string** like `'1'`. Two
test-side consumers fed that value straight into tensor/device
placement:

- `reduce_boolean_flags` in `tests/unit/common.py` (backbone of
`allclose_on_all_ranks`, the "all ranks succeed or fail together" check)
- 15 call sites in `tests/unit/v1/autotp/test_autotp_training.py`

On CPU this fails immediately with `RuntimeError: Invalid device string:
'1'` — before the first collective even runs.

## Change

- Use `current_device_name()`, which returns a full device string on
every backend (`'cpu'`, `'cuda:N'`, `'mps:0'`, …) and is equivalent to
the index on GPU backends.
- In `reduce_boolean_flags`, carry the flag in a 1-dim tensor: gloo
rejects 0-dim inputs to `all_gather_into_tensor` (NCCL tolerates them),
so the previous form would have failed on the very next line.

## Validation

Validated as part of the multi-rank CPU CI experiment in deepspeedai#8381
(same-commit baseline comparison): this failure class disappeared, zero
regressions on previously-passing tests.

---------

Signed-off-by: Guokai Ma <guokai.ma@intel.com>
Signed-off-by: Ma, Guokai <guokai.ma@intel.com>